Rule Weight Optimization and Feature Selection in Fuzzy Systems with Sparsity Contraints

In this paper, we are dealing with a novel data-driven learning method (SparseFIS) for Takagi-Sugeno fuzzy systems, extended by including rule weights. Our learning method consists of three phases: the first phase conducts a clustering process in the input/output feature space with iterative vector quantization. Hereby, the number of clusters = rules is pre-defined and denotes a kind of upper bound on a reasonable granularity. The second phase optimize the rule weights in the fuzzy systems with respect to least squares error measure by applying a sparsity-constrained steepest descent optimization procedure. This is done in a coherent optimization procedure together with elicitation of consequent parameters. Depending on the sparsity threshold, more or less rules weights can be forced towards 0, switching off some rules. In this sense, a rule selection is achieved. The third phase estimates the linear consequent parameters by a regularized sparsity constrained optimization procedure for each rule separately (local learning approach). Sparsity constraints are applied here in order to force linear parameters to be 0, triggering a feature selection mechanism per rule. In some cases, this may also yield a global feature selection, whenever the linear parameters of some features in each rule are near 0. The method is evaluated based on high-dimensional data from industrial processes and based on benchmark data sets from the internet and compared to wellknown batch training methods in terms of accuracy and complexity of the fuzzy systems. Keywords— Takagi-Sugeno fuzzy systems, iterative vector quantization, rule weight optimization, sparsity constraints, rule selection, embedded feature selection